Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



An Analysis of LRE Placement Data Collection 

for Region 1: 

Verification, Consistency, and Comparability 



Bruce Bull 
EDSIS 

Kristin Reedy, Ed.D. 



NORTHEAST REUONAI 



NERRC 



K E s U IK C E CENTER 



Northeast Regional Resource Center 
Learning Innovations at WestEd 
20 Winter Sport Lane 
Williston, VT 05495 



January 2007 



Region 1 2006 Westat State Analysis LRE Grant: Final Report 



Northeast Regional Resource Center 



1 




Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



An Analysis of LRE Placement Data Collection for Region 1: 
Verification, Consistency, and Comparability 

Final Report 



Table of Contents 

ABSTRACT 3 

EXECUTIVE SUMMARY 4 

BACKGROUND INFORMATION 6 

POLICY ISSUE ADDRESSED 7 

METHOD 7 

RESULTS 1 1 

DISCUSSION 23 



TABLE 1 
TABLE 2 

TABLE 3 
TABLE 4 

TABLE 5 

TABLE 6 

FIGURE 1 
FIGURE 2 



Number of useable surveys by State 8 

Employment titles/positions of survey respondents 8 

State respondents knowledge of TA and training availability 12 

Percent of respondents who used TA and their perceived value of TA 
used 1 



Mean frequency of using different assists when determining difficult 
to code LRE situations (5-point scale 1 -never to 5-always) 

State used school age placement definitions of educational 
environments of children with disabilities ages 6-21 

Student scenario LRE classifications grouped by State 15 

State classifications by scenario 19 



APPENDIX A NORTHEAST STATES LRE DATA 33 

APPENDIX B CHECKLIST OF LRE MATERIALS 35 

APPENDIX C PROCESS FORM 36 

APPENDIX D GENERIC SURVEY INSTRUMENT 37 

APPENDIX E TA AND TRAINING RESULTS 42 

APPENDIX F STATE SCHOOL AGE LRE DEFINITIONS 43 

APPENDIX G AGGREGATE AND STATE LEVEL CONSENSUS 

ON CODING LRE FOR SCHOOL AGE SCENARIOS 48 



Northeast Regional Resource Center 



7 



Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



An Analysis of LRE Placement Data Collection for Region 1: 
Verification, Consistency, and Comparability 

Final Report 



ABSTRACT 

Historically, states in the northeast region have differed considerably from neighboring 
states in the percentage of their special education student population served in different 
educational environments. Placement data indicate the degree to which students with 
disabilities are placed in the least restrictive environment (LRE) along the continuum of 
placement options. This study investigated the degree of consistency with which 
placement classifications are used by different raters or coders within and across states 
within the region and explored possible explanations for state-to-state variation in 
placement patterns. State-specific surveys were created and each contained a core set 
of questions and exact scenarios. State respondents (special education local district 
personnel) independently assigned an LRE placement classification to the fictitious 
students in scenarios based on state-provided LRE placement descriptions. Additional 
questions were asked about availability, utilization, and perceived value of state- 
delivered training and technical assistance relative to LRE knowledge transfer. Wide 
variation existed in the scenario response patterns of respondents both within states 
and across states. Local, regional, and national recommendations are provided for 
improving the reliability of LRE data. 
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EXECUTIVE SUMMARY 

States are required to report placement data for students with disabilities served under 
the Individuals with Disabilities Education Act (IDEA) to the U.S. Office of Special 
Education Programs (OSEP) on an annual basis. Placement data indicate the degree to 
which students with disabilities are placed in the least restrictive environment (LRE) 
along the continuum of placement options. LRE data indicate the environment in which 
special education services are provided and are often used to determine the likelihood 
of intervention by the State Education Agency (SEA) to the local level. Placement data 
may also be used to determine levels of intervention by OSEP at the state level. In the 
northeast, a large amount of variation exists across states with respect to IDEA Part B 
LRE data. The purpose of this study was to determine if circumstances other than 
differences in service delivery models might affect LRE data within states in the 
northeast region. The study focused on placement data for school age students. 

Three specific policy questions were addressed regarding LRE data and a fourth policy 
question was added during the study. The additional question addressed the 
availability, utilization, and perceived value of various LRE technical assistance (TA) 
and training opportunities that SEAs make available to local education agencies (LEAs). 
The policy questions investigated were: 

1 . To what degree are state directions for reporting/counting children in specific 
placement categories consistent with OSEP directions? 

2. To what degree are state directions to LEAs for reporting/counting children in 
specific placement categories consistent across states? 

3. How do LEAs’ understanding of those directions and the actual reporting of 
data compare or are comparable across LEAs within and across states? 

4. What types of technical assistance and training are available to LEAs on 
classifying LRE placement information and what is their level of utilization and 
perceived value? 

SEA Directors of Special Education in the northeast authorized the Northeast Regional 
Resource Center (NERRC) to initiate this study on their behalf. NERRC, in turn, 
contracted with an independent consultant to do the work. The consultant was invited to 
meet with SEA Directors before involving state IDEA Data Managers. Data Managers 
provided state information relative to LRE placement, provided input on difficult to code 
school age special education scenarios, and outlined the types of technical assistance 
and training provided by their state. The consultant worked closely with six of the eight 
northeastern states to create a state-specific online survey. Two states chose not to 
participate in the survey. 

The consultant created and housed the online surveys. Section one of the survey 
focused on the availability, utilization, and perceived value of different types of technical 
assistance and trainings an SEA might make available. Section two consisted of LRE 
scenarios for school age Part B students. Respondents were asked to categorize each 
placement based on state provided LRE placement descriptions. The URL for each 
online survey was provided to each participating SEA. Without OMB clearance, federal 
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funding of this study restricted the number of LEA surveys administered to nine per 
state, and the nine LEAs were not randomly selected. The six participating states 
generated 45 useable surveys from a total possible of 54 (nine surveys across six 
states), a response rate of 83 percent. Therefore, this sample is not representative of 
any state or the northeast region. Selected findings are included below. Survey 
respondents were predominately local Special Education Directors, with an average 
tenure in their position of over 10 years. They generally scored themselves as “highly 
confident” in the LRE data their district reported to the state. 

Respondents in most states were inconsistent with respect to their knowledge of what 
LRE technical assistance and trainings were available from their SEA. Across all of the 
states, respondents within a state disagreed on the very existence (or nonexistence) of 
the 1 1 technical assistance and training methodologies in exactly half of the cases. (See 
Table 3.) Respondents placed more value on technical assistance methodologies that 
were one-on-one or interactive in nature, rather than group trainings and static 
documents. The one exception was a Data Dictionary that five respondents indicated 
they had access to and used. The mean score calculated from these five respondents 
on use of the Data Dictionary was higher than all other methods of imparting knowledge 
about reporting LRE placement data. 

Large variation existed in the way respondents classified the same special education 
scenarios. Regionally there was inconsistency in the way respondents classified the 
same scenario. Moreover, except for the respondents from the state of New York, there 
was inconsistency in the way respondents within a state classified the same scenario. 

In the review of state LRE definitions, two states were found to have modified the 
language of the OSEP LRE definitions. New Jersey adopted the OSEP-proposed 2006 
definitions one year early. This was believed to have had no impact on LRE placement 
determinations or understanding of placement definitions. Massachusetts, however, 
included the terms “Full Inclusion, Partial Inclusion, and Substantially Separate 
Classroom” within the three OSEP placement definitions based on time in the 
regular/general education classroom. It was hypothesized that these descriptors might 
increase opportunities for miscoding students for this state. A limited review of 
Massachusetts’s LRE data suggests this possibility may be worth further exploration. 

The existence of large variation in LRE data at the LEA level within a state suggests 
that states may be prudent to thoroughly investigate their LRE data. Moreover, steps 
should be taken to improve technical assistance and supports to increase the reliability 
of the reporting of local LRE data. Recommendations are provided to assist special 
education administrators, at multiple levels, with the collection and reporting of accurate 
LRE data. 



This work was funded by the Office of Special Education Programs via a Westat 2006 State 
Analysis Grant. The Northeast Regional Resource Center was the successful proposer and hired 
Bruce Bull of Education Data Services and Information Systems to conduct the work. 
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BACKGROUND INFORMATION 

Data on the educational environments in which students with disabilities served under 
IDEA receive special education are classified, collected, reported, and often used at the 
national, state, and local level as an important indicator of special education service 
delivery. These data, referred to as educational environment, placement, or in this 
report Least Restrictive Environment (LRE) data, are sometimes used as a measure of 
the degree to which students with disabilities have access to the general education 
curriculum and are being educated in the least restrictive environment. These data are 
used by states and by the Office of Special Education Programs (OSEP) for important 
and consequential purposes. At the national level, states have been ranked by 
educational environment data as a way of targeting states for OSEP-directed technical 
assistance. At the state level, local district LRE data are sometimes used to determine 
state-to-local focused monitoring. Now placement data are required in states' State 
Performance Plans (SPPs) and Annual Performance Reports (APRs) and will be used 
by OSEP to make state determinations under Section 616 of IDEA 2004. Each State 
must collect valid and reliable information as needed to report annually to OSEP on the 
indicators established by the U.S. Secretary of Education for the State Performance 
Plans [34 CFR 300.601(b) and 20 U.S.C. 1416(b)], OSEP annually reviews each state’s 
performance report and makes a determination of the state’s relative need for 
assistance and/or intervention in implementing the requirements of Part B of the Act. 

Those not familiar with these high stakes LRE data might assume LRE placement 
definitions are uniformly understood and consistently applied when reporting LRE data 
to the state and subsequently to the federal level. However, there is a considerable 
degree of variability across states on the percentages of students reported to be 
receiving special education services in various LRE placement categories. Reviewing 
the IDEA Annual Reports to Congress or the data available on the www.ideadata.org 
Web site attests to state-to-state variation in LRE data. Much of this variation is due to 
real differences in the ways special education and related services are provided in 
different states. For example, the percentage of students served in private schools 
differs significantly across states across the U.S. (Appendix A has LRE differences for 
northeast states.) 

In addition to state level differences, experienced IDEA Data Managers, SEA Directors 
of Special Education, SPP writers, and OSEP personnel, among others, can attest to 
widely varying LRE data among districts within a single state. Different placement 
descriptions, different levels of definitional understanding, challenges in calculating time 
in the general education classroom, attendance (or lack of) at instructional trainings, 
available resource materials, etc. — as well as differences in special education 
delivery — are “areas of interest” in looking at LRE variation within a state. 

Knowing that cross-state differences in service delivery do exist, this study attempted to 
ascertain if factors, other than differences in service delivery models, might be 
contributing to the wide LRE variation demonstrated across and within the northeast 
states. The existence of variation in LRE data at the LEA and SEA level suggests that 
states may be prudent to similarly investigate the accuracy and inter-rater reliability of 
their LRE data. The focus of this study was Part B LRE data for the school age 
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population served under IDEA. The study did not address preschool children reported 
under Section 619. 



POLICY ISSUES ADDRESSED 

For this study, three specific policy questions were addressed regarding LRE data 
collection in states within the northeast region. 

1 . To what degree are state directions for reporting/counting children in specific 
placement categories consistent with OSEP directions? 

2. To what degree are state directions to LEAs for reporting/counting children in 
specific placement categories consistent across states? 

3. How do LEAs’ understanding of those directions and the actual reporting of 
data compare or are comparable across LEAs within and across states? 

Additionally, through the course of the investigation, a fourth policy question was 
developed and addressed: 

4. What types of technical assistance and training are available to LEAs on 
classifying LRE placement information and what is their level of utilization and 
perceived value? 

Answers to these questions are expected to help the region as a whole and specific 
SEAs within the northeast understand the reasons behind some of the variation in their 
LRE data. 



METHOD 

This section describes how information was gathered and used in this study. Details are 
included on: participants, materials used, and procedures followed to gather the desired 
LRE data. 

Participants 

The eight states of Region 1 (CT, ME, MA, NH, NJ, NY, Rl, and VT) participated in this 
study. All eight states provided information about their policies for communicating LRE 
data collection information to their LEAs. However, only six of the eight states 
participated in the LEA survey that captured district level information on LRE. Within 
the confines of this federally funded study each participating state could have a 
maximum of nine LEA respondents to their state’s survey. Two states were unable to 
secure nine completed surveys, and other states had instances where a respondent’s 
survey was not adequately completed and could not be used. A total of 45 usable 
surveys were gathered from 54 possible across the six states (83%). Table 1 provides 
detail on the completed surveys by each state. 
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Table 1. Number of useable surveys by State 


State 


CT 


MA 


ME 


NH 


NJ 


NY 


Rl 


VT 


Total Surveys 


9 


9 


6 


9 


6 


9 


NA 


NA 


Usable Surveys 


9 


8 


6* 


8 


6 


8 


NA 


NA 


Total 


45 Useal 


ble Surveys from 6 States 


*Two ME surveys were partially completed and were used whenever possible. 



As shown in Table 2, the vast majority of survey respondents were local Special 
Education Directors (or the equivalent based on state preferred title). 



Table 2. Employment titles/positions of survey respondents 


Assistant Director of Special Education 


1 


Assistant Superintendent of Student Services 


1 


Committee on Special Education (CSE) Chairperson 


2 


Executive Director of Student Services and Special Education 


1 


Special Ed Secretary/Administrative Assistant /Child Count Data Entry Clerk 


3 


Special Education Director or Director of Special Education 


37 


Totals 


45 



Materials 

This study gathered information through paper forms and electronic surveys. Each item 
is described below. 

1 . Checklist of LRE Materials (Appendix B) This form, usually completed by the 
Part B Data Manager, provided insight into the types of training/technical 
assistance states provided to their LEAs regarding the collection of LRE data. 
This information was later used in the LEA surveys. 

2. Process Form (Appendix C) Completed onsite by the state Data Managers at 
the OSEP / OSEP/Westat 2006 Overlapping Part B & Part C Data Meeting, this 
form provided the consultant with more insight into the processes used by each 
SEA relative to the collection of IDEA 618 data. Answers to specific LRE data 
collection questions were provided. 

3. Difficult Scenarios Shared Based on their experience Data Managers were 
asked to provide the consultant with two examples of real scenarios that were 
difficult to determine the correct LRE category. Most Data Managers provided 
this to the consultant during the face-to-face meeting. 

4. Select Survey Tool SelectSurvey.NET is the software tool the consultant used 
for the creation of the online surveys. The consultant created individual state 
surveys, which were housed on the consultant’s server and formed the 
foundation for the collection of data used in this study. Each Data Manager 
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reviewed and approved their state’s survey online before sharing the state- 
specific survey URL with their potential LEA respondents. 

Each state survey consisted of two sets of questions. The first set inquired about 
the availability and usefulness of technical assistance and trainings designed to 
transfer LRE data knowledge from the SEA to the LEAs. Methods included for 
example, a Data Dictionary, hands-on-training, Frequently Asked Questions (FAQs) 
documents, availability of an SEA expert, etc. The second set of questions on each 
state survey contained special education placement scenarios. Respondents were 
asked to code these scenarios based on OSEP-established (and sometimes SEA- 
refined) LRE category definitions. See Appendix D for a blank copy of the generic 
survey instrument. The survey instrument contains the content used across the six 
state surveys. 

Note that each state survey was unique. States were allowed to include additional 
scenarios, state-specific language, and add technical assistance/training activities, 
to make the survey as meaningful as possible for their state. State-specific survey 
modifications were also done to comply with the respondent restriction required of 
this study. Specifically, this was a federally funded study and therefore a maximum 
of nine respondents per survey was permitted in order to comply with federal 
paperwork burden policies and information gathering restrictions. Creating unique 
surveys for each state maximized the total number of potential respondents. 

Procedures 

The following are the procedures used in this study and the approximate date of the 

procedure. 

1 . Fall 2005 Prior to the proposal being submitted, each SEA wrote a letter of support 
for the project. In most — but not all — cases, this support translated to involvement 
for the entirety of this study. 

2. March - November 2006 The contracted consultant and NERRC maintained 
frequent communication prior to and throughout the study. Ongoing emails and 
every other week teleconferences developed and maintained excellent 
communication. This was most helpful when small obstacles arose. At those points, 
NERRC knew the background and together the consultant and NERRC formulated 
immediate and effective solutions. 

3. April 2006 The consultant was invited and attended a regional meeting to describe 
the project to the northeast SEA Directors. He provided an overview of the study 
including the expected level of involvement from each state. The consultant asked 
SEA Directors to share information and handouts with state Part B Data Managers. 
As designed, after that regional meeting, most SEA project communication occurred 
between the consultant and the state IDEA Part B Data Managers. 

4. May 2006 The consultant held a 1 -hour face-to-face introduction and work session 
with Part B Data Managers from the eight northeast states at the NERRC-hosted 
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luncheon held in conjunction with the OSEP/Westat 2006 Overlapping Part B & Part 
C Data Meeting. After an introduction to the study, each IDEA Data Manager 
completed forms describing their SEA’s process and materials (e.g., definitions, 
trainings) used to collect and report LRE data. IDEA Data Managers were also 
asked to provide examples of “challenging to code” scenarios. These were used 
later in the state surveys. (See Appendix B Checklist of LRE Materials and 
Appendix C Process Form.) 

5. May - June 2006 The consultant drafted scenarios that might ideally work in 
all/most states in the study. Draft scenarios were shared with NERRC and Data 
Managers. 

6. May - June 2006 The consultant drafted an additional section of the survey to 
collect information on SEA training and technical assistance associated with LRE 
data collection. 

7. May - August 2006 Based on each state’s readiness, the consultant developed 
draft state-specific surveys based on established scenarios and additional state 
input. Data Managers reviewed survey drafts and provided feedback. The draft and 
review process was repeated until each state-specific survey was acceptable to that 
state. The consultant and SEA Data Manager tested and reviewed an online draft of 
that state’s survey. The consultant made changes when necessary to the online 
survey draft. Once finalized, the consultant removed all practice data and sent the 
URL link to SEA Data Manager. 

8. June - August 2006 The consultant developed and provided states with draft 
documents to be used by SEAs when introducing the study to potential LEA 
respondents. 

9. June - September 2006 SEAs sent information and the URL to potential LEA 
respondents. Each state selected the LEA respondents from their state. In some 
cases, SEAs solicited only nine LEAs. In other cases SEAs expanded the 
solicitation to more than nine knowing the software was designed to close the survey 
after nine responses had been received. Data Managers shared in conversation 
that in many cases they targeted LEAs that they anticipated would be more 
“responsive” (that is, had a higher likelihood of following through on the SEA 
request). Given the non-representational limits of the study, the convenience 
sample methodology was an accepted method of including LEAs as respondents in 
the study. All data from LEAs were collected and reported anonymously and directly 
to the consultant’s server. All responses affiliated with a particular state remained 
anonymous at the LEA level. 

10. June - September 2006 The consultant maintained communication with Data 
Managers keeping them apprised of respondent status. (Frequently, the consultant 
encouraged additional SEA communication to LEAs to increase respondent 
numbers.) 
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1 1 .June - September 2006 Once the maximum number of respondents was reached, 
the survey was programmed to stop accepting additional data. 

1 2. September - December 2006 The consultant removed, aggregated, and cleaned 
data and conducted analysis, drafted the report, and reviewed report with NERRC 
and Westat. The report was revised based on Westat feedback and a final version 
was submitted. 



RESULTS 



Respondents 

Survey respondents had, on average, 10.7 years of experience in their current position. 
When asked, ’’What is your level of confidence in the accuracy of the data you are 
reporting to your SEA on the placement classifications of students with disabilities?” 
respondents overall reported a high confidence rating (4.4 on a 5-point scale). 

LRE Training and Technical Assistance 

The survey (Appendix D) asked respondents their a) knowledge of the availability, b) 
engagement with, and c) perceived value of, different types of SEA training and 
technical assistance methods associated with LRE knowledge transfer. Detailed 
technical assistance and training data by state and multi-state are in Appendix E. 

Respondents within the same state were often inconsistent in their knowledge of the 
availability of different types of LRE data collection training and technical assistance 
provided in their state. Table 3 provides a summary of the number of respondents who 
believed a particular type of assistance existed (availability) out of the total number of 
respondents who answered that question from that state. Shaded cells indicate that 
there were consistent responses within a state. No shading indicates a lack of 
respondent consistency as to the existence of an assist. Consistency was defined as 
all, or all but one, of the respondents within a state indicating that the assist existed or 
did not exist. Eleven types of technical assistance across six states results in a total of 
66 “cases” or cells. Thirty-three of the 66 cells are shaded showing that in half of the 
cases, there was agreement within the state that the type of technical assistance was 
available. In the other 33 cases, half the measures, state respondents were inconsistent 
in their knowledge of the existence of LRE technical assistance and trainings. 

The overall rate of consistency within states of respondents’ reported understanding of 
the existence or non-existence of different forms of TA within the state ranged from a 
low of 18% in Massachusetts to a high of 73% in New Jersey. 



1 1 
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Table 3. State respondents knowledg 


ie of TA and training availal 


bility * 


Type of Training or Technical 
Assistance 


CT 


MA 


ME 


NH 


NJ 


NY 


1 . Data Dictionary 


2 of 9 


1 of 8 


0 of 4 


4 of 8 


2 of 6 


4 of 8 


2. Expert Available Via Phone 


9 of 9 


5 of 8 


3 of 4 


7 of 8 


6 of 6 


7 of 8 


3. Expert Available Via E-mail 


9 of 9 


5 of 8 


3 of 4 


7 of 8 


5 of 6 


5 of 8 


4. Colleague Available 


9 of 9 


5 of 8 


2 of 4 


7 of 8 


6 of 6 


7 of 8 


5. Training Video 


2 of 9 


2 of 8 


1 of 6 


0 of 8 


0 of 6 


0 of 8 


6. Frequently Asked Questions and 
Answers Posted 


5 of 9 


5 of 8 


1 of 4 


2 of 8 


5 of 6 


5 of 8 


7. Documents (other than Data 
Dictionary) 


9 of 9 


4 of 8 


2 of 4 


7 of 8 


6 of 6 


8 of 8 


8. Face-to-face Training 


9 of 9 


6 of 8 


3 of 6 


5 of 8 


4 of 6 


6 of 8 


9. Hands-on Training 


2 of 9 


4 of 8 


2 of 6 


4 of 8 


2 of 6 


2 of 8 


10. Automated Help Inside an 
Application 


2 of 9 


3 of 8 


3 of 5 


1 of 8 


1 of 6 


4 of 8 


1 1 . Distance Learning 


0 of 9 


1 of 8 


3 of 6 


1 of 8 


1 of 6 


1 of 8 


Summary of state respondents’ 
agreement that all or all but one 
respondent agreed a TA type 
existed or did not exist. 


6/11 

( 55 %) 


2/11 

( 18 %) 


5/11 

( 45 %) 


7/11 

( 64 %) 


8/11 

( 73 %) 


5/11 

( 45 %) 


* Shaded cells indicate consistent responses within a state, defined as all or all but one of 
the respondents within a state indicating that the assist existed or did not exist. 



To ascertain a measure of utilization or ’’engagement with” technical assistance, the 
survey asked respondents who indicated that a technical assistance or training existed, 
if they had used (engaged) that particular type of assist in the last 24 months. Those 
who had engaged with the assist were asked to assign a value to that technical 
assistance. Table 4 displays the percentage of those engaging with each type of 
technical assistance or training and the mean value they assigned to the assist. 
(Technical assistance and training are displayed in descending order based on mean 
values.) The availability of a Data Dictionary is the most highly valued form of technical 
assistance, as reported by the five respondents who used it. Based on a frequency 
count, the three most frequently reported available forms of TA were “Expert Available 
Via Phone” (n= 37), “Colleague Available” (n= 36) and “Documents other than Data 
Dictionary” (n= 36). Utilization is the percent of respondents that reported using an 
available assist. The two forms of technical assistance with the highest reported 
utilization are Colleague Available” (83%) and “Frequently Asked Questions and 
Answers” (83%). 
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Table 4. Percent of respondents who used TA and their perceived value of TA used 


Type of Training or 


Respondents who Used/Engaged the TA 


Mean Value 


Technical Assistance 


Of Those Saying it was Available 


1 (low) - 5 
(high) 


Data Dictionary 


5 used of the 13 saying was available (38%) 


4.80 (n = 5) 


Expert Available Via Phone 


28 used of the 37 saying was available (76%) 


4.39 (n =28) 


Expert Available Via E-mail 


26 used of the 34 saying was available (76%) 


4.27 (n = 26) 


Colleague Available 


30 used of the 36 saying was available (83%) 


4.27 (n = 30) 


Training Video 


1 used of the 5 saying was available (20%) 


4.00 (n = 1) 


Frequently Asked Questions 
and Answers 


19 used of the 23 saying was available (83%) 


4.00 (n = 19) 


Documents (other than Data 
Dictionary) 


27 used of the 36 saying was available (75%) 


3.85 (n = 27) 


Face-to-face Training 


23 used of the 33 saying was available (70%) 


3.83 (n = 23) 


Hands-on Training 


12 used of the 16 saying was available (75%) 


3.67 (n = 12) 


Automated Help Inside an 
Application 


7 used of the 14 saying was available (50%) 


3.43 (n = 7) 


Distance Learning 


2 used of the 7 saying was available (29%) 


3.00 (n = 2) 



After being asked about their knowledge, use and perceived value of different types of 
training and TA, respondents were asked to rate the frequency that they used each of 
four different methods when faced with a difficult to code LRE scenario. (See Table 5.) 
The highest mean rating (3.60) was earned for “Reviewing TA Materials,” compared 
with “Take Best Guess,” which had the lowest mean rating of 2.42. 



Table 5. Mean frequency of using different assists when determining difficult to 
code LRE situations (5-point scale 1 -never to 5-always) 


Take Best 
Guess 


Call Someone at 
the SEA 


Ask Colleague 


Review TA 
Materials 


2.42 


3.35 


3.49 


3.60 



State Definitions 

A review was conducted of each state’s LRE definitions. (See Appendix F for complete 
definitions from northeast states.) Most states used the OSEP language as provided in 
OSEP’s Section 618 report forms. However two states that were surveyed modified the 
language defining the different LRE placements. New Jersey used (2005-06) the 
OSEP-proposed Part B language for the 2006-07 school year. Massachusetts added 
descriptors (full inclusion, partial inclusion, and substantially separate classroom) to the 
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time outside regular class placement categories. Procedurally, each state’s definitions 
were included verbatim in their state specific survey. Definitional differences are shown 
by state in Table 6. 



Table 6. State used school age 
of children with disabil 


i placement definitions of educational environments 
ities ages 6-21 


Key 


OSEP Definitions 
NH, ME, NY, CT 


NJ Language 


MA Language 


<21 


Special education 
outside regular class less 
than 21 percent of day 


Included in general ed 
setting at least 80% of 
day 


All aqes, Full Inclusion 
Special education services 
outside the General 
Education Classroom less 
than 21% of the time 


21-60 


Special education 
outside regular class at 
least 21 percent of day 
and no more than 60 
percent of day 


Included in general ed 
between 40% and 80% 
of day 


All aqes, Partial Inclusion 
Special education services 
outside the General 
Education Classroom 21% 
to 60% of the time 


> 60 


Special education 
outside regular class 
more than 60 percent of 
day. 


Included in general ed 
up to 39% of the day 


All aqes, Substantially 
Separate Classroom Special 
education services outside 
the General Education 
Classroom more than 60% 
of the time 


PuS 


Public separate school 


Same as OSEP 


All ages, Public Separate 
Day School 


PrS 


Private separate school 


Same as OSEP 


All ages, Private Separate 
Day School 


PuR 


Public residential facility 


Same as OSEP 


All ages, Public Residential 
Institutional Facilities 


PrR 


Private residential facility 


Same as OSEP 


All ages, Residential School 


H 


Homebound/Hospital 


Same as OSEP 


Homebound/Hospital 



Categorization of School-Age Scenarios 

Throughout this study the question, “What is the ‘correct’ answer?” was asked by 
respondents. The study was not, however, designed to determine the correctness of a 
particular scenario classification. Rather, the study measured state and regional 
respondent differences in classifying LRE placement scenarios. To measure this 
variation, the last part of the survey queried respondents on their categorization of 
school age special education LRE scenarios. Respondents were asked to classify the 
placements based on their own state-specific LRE definitions. (State-specific scenarios, 
included in some state surveys, are not reported in this report.) 

These classification findings demonstrate inconsistency both across the region and 
within each state when classifying the same scenario. Results are presented here in 
two figures. Data are first viewed in scenario-specific horizontal stacked bars (Figure 
1 ). The text of each scenario is presented. To the right of the scenario text is a graph 
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depicting the percentages of LRE placement responses reported for that scenario by 
each surveyed state. The responses from each state are shown on one horizontal bar. 
Additionally, the top bar in each graph (labeled All) shows the aggregate of all study 
respondents for that scenario. The purpose of this view is to show the differences and 
similarities across states with respect to each scenario, with a particular emphasis on 
regional variation. 

Perfect agreement would be each bar for each state having the same color/pattern. 
This would indicate that respondents across states in the study classified the same 
scenario with the same LRE placement code. 



Figure 1 . Student scenario LRE classifications grouped by State 




LRE Classification Key 

Outside regular class < 21 % of Day 
Outside regular class 21-60 % of Day 
Outside regular class > 60% of Day 
Public separate school 
Private separate school 
Public residential facility 
Private residential facility 
Don’t know 



Steve 



Steve Scenario Classifications 



Steve’s parents removed Steve, 
a 7th grader, from public school 
and placed him in a private prep 
school in their district. Later 
Steve is found eligible for 
special education. Steve now 
receives 2 hours a day of pull- 
out services in the resource 
room at the private school. 
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Jamie 



Jamie Scenario Classifications 



Jamie goes to the local high 
school in the morning for 15 hours 
a week. While there Jamie 
receives 5 hours a week of special 
education in a co-taught program. 
(The co-taught classroom has one 
special education teacher and one 
general education teacher. The 
general education teacher is the 
teacher of record for this 
classroom.) In the afternoon 
Jamie goes to a work study 
program at the local McDonald’s 
for a total of 15 hours a week. 




40% 60% 



80% 100% 



Holly 1 



(Holly 1) A district places Holly at 
a residential school for students 
with sensory impairments. Holly 
lives close to the school, so she 
takes the bus to and from school 
as a day student. Assume this is a 
public residential school. 



Holly 1 Scenario Classifications 
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Holly 2 



Holly 2 Scenario Classifications 



(Holly 2) What is Holly's 
placement if the school is a private 
residential school? 




0% 20% 40% 60% 80% 100°/ 



Pat 



Pat Scenario Classifications 



The district placed Pat at a private 
separate school for students on 
lEPs. Though still attending there 
part of the day, he is now 
transitioning back to his local high 
school. Currently his daily routine 
is to attend the private school from 
9-12 then he is bussed to the 
public high school. In the 
afternoon he is in two general 
education classes with an 
instructional assistant who is 
responsible for Pat and another 
student with disabilities. He also 
attends one special education life 
skills class at the public high 
school. Each high school class is 
an hour long. 



All 

NY 

NJ 

NH 

MA 

ME 

CT 



— r- 






INI 

INI 


lllllll^ 


■ 
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ITT 




1 


1 1 
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Jane 



Jane Scenario Classifications 



Jane is a parentally placed private 
school student with a Service 
Plan. During the first part of the 
year she received 1 hour of 
speech/week on a “pull-out” basis. 
The district decided to put her on 
an IEP in November but service 
levels and placement remain the 
same. 



All 
NY 
NJ 
NH 
MA 
ME 
CT 

0% 20% 40% 60% 80% 100% 



NJ did not 


score Jane s 


cenario 








llllllllllllll 
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Matt 



Matt Scenario Classifications 



Matt has two classes in community 
job experience, two general education 
content courses with special 
education support as needed, one 
class in a resource room with a 
special education teacher on study 
skills, and one elective class with no 
additional support. He has no other 
classes and the frequency and 
duration of all classes is the same. 
One of the community job experience 
classes is a general ed class, the 
other is a special education class. 
Also, in one general education class 
Matt is counted on enrollment by the 
special education teacher, in the other 
class the regular ed teacher counts 
him for attendance. 



All 

NY 

NJ 

NH 

MA 

ME 

CT 




0 % 



20 % 



40 % 



60 % 



80 % 



100 % 



As can be seen in Figure 1 , respondents across the states were inconsistent in how 
they coded these difficult scenarios. None of the scenarios show high levels of 
agreement across the states. For example, for the Holly 2 scenario, five different 
placement categories were used with varying degrees across state respondents. There 
were no scenarios in which states were in agreement on the appropriate placement 
category. However, some scenarios show similar variation patterns. 
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The next set of graphs (Figure 2) displays the same data in vertical bar graphs with a 
state focus. Each vertical bar graph shows how the respondents from a given state 
classified each scenario. The purpose of this view is to show the state specific variation 
or patterns of LRE placement classifications. 



Figure 2. State classifications by scenario 

LRE Classification Key 



100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 

Steve Jamie Holly 1 Holly 2 Pat Jane Matt 



CT Scenario Classifications 




Outside regular class < 21 % of Day 
Outside regular class 21-60 % of Day 
Outside regular class > 60% of Day 
Public separate school 
Private separate school 
Public residential facility 
Private residential facility 
Don’t know 



Northeast. Rpfrinn/il Rpanurcp C.pnlpv 



19 






Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



MA Scenario Classifications 




ME Scenario Classifications 
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NY Scenario Classifications 




Again, ideally the bars would be a single color for each scenario indicating agreement 
among state respondents on each scenario. The more colors in the stacked bar, the 
less agreement existed. Except for the state of NY, there were differences within each 
state on every scenario. In New York, there was unanimity on the classification for four 
of the seven scenarios: Jamie, Holly 1 , Holly 2, and Jane. No other state demonstrated 
100% agreement on any of the scenarios. 

Figures 1 and 2 show very little agreement among respondents in how scenarios were 
classified. With the exception of New York, within state agreement was essentially non- 
existent. Some classification patterns do exist across states, suggesting that 
respondents across states have similar challenges when classifying difficult to code 
placements. Appendix G contains a data table with further analyses of the scenario 
classification data. 

Finally, at the very end of the survey respondents were given the opportunity to provide 
feedback or comments. Three general comments represented a broader perspective 
and have potential implications for data collection practices in other sates, as noted 
below: 

1 . These are the (types of) examples that I find helpful for directors or 
supervisors to have correct responses. 

2. I think the tendency is (to) err on the side of being more conservative when 
there is doubt. It would be so much easier if rather than a percentage, the 
State provide the number of minutes (i.e., 300 minutes or less per week vs. 
21% or less, etc.). 

3. More training needs to be offered ... at least more specific resources on the 
DOE site. 
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DISCUSSION 

The purpose of this study was to determine if circumstances other than differences in 
service delivery models might affect LRE data within and across states in the northeast 
region. The study focused on LRE placement classifications for school age IDEA Part B 
students. 

Three specific policy questions were addressed regarding LRE data and a fourth policy 
question was added during the study. The additional question addressed the 
availability, utility and perceived value of the various types of LRE technical assistance 
and training SEAs make available to LEAs. The policy questions investigated were: 

1 . To what degree are state directions for reporting/counting children in specific 
placement categories consistent with OSEP directions? 

2. To what degree are state directions to LEAs for reporting/counting children in 
specific placement categories consistent across states? 

3. How do LEAs’ understanding of those directions and the actual reporting of 
data compare or are comparable across LEAs within and across states? 

4. What types of technical assistance and training are available to LEAs on 
classifying LRE placement information and what is their level of utilization and 
perceived value? 

The findings of this study suggest that there are very real differences in how both LEAs 
within a state and LEAs across states interpret and classify the same school age special 
education LRE scenarios. It is the overall finding of this study that inter-rater reliability 
issues, rather than cross-state differences in special education delivery models, 
contribute to the variation in LRE data exhibited in the northeast. The remainder of this 
section will acknowledge the limitations of the study, highlight important results in light 
of the policy issues addressed, provide recommendations for future studies in this area, 
and most importantly provide recommendations to be considered at the state, regional, 
and national level to address or alleviate potential inter-rater reliability issues concerning 
LRE data. 

Limitations 

The following limitations affected this study, listed in descending order of influence. 

First and foremost, the study restricted the number of responses that could be collected. 
Federal funding of this study, without OMB clearance, restricted the number of LEA 
surveys administered to nine per state, therefore the sample was not representative of 
any state or the northeast region. Once this restriction was realized, the study design 
was modified to maximize the number of respondents that could be included. This was 
done by creating unique surveys for each state. However, the resulting sample was too 
small to draw any definitive state or regional conclusions. 

Second, the respondent group itself may not have been reflective of those most 
involved in categorizing and reporting LRE data. Although the survey was designed for 
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anyone associated with LRE data determination and transference, 37 out of 45 (82%) of 
respondents were local Special Education Directors. In many cases, local Special 
Education Directors are not involved with the day-to-day student-level data issues such 
as LRE classification. Although the respondent group had more than 10 years average 
tenure in their current positions and self-reported a high confidence level in the 
accuracy of LRE data submitted to their SEA, their actual level of involvement with LRE 
data and determining LRE classification was not known. It could be that their 
administrative position limits their awareness of the nuances sometimes associated with 
classifying such data. The predominance of local Special Education Directors as 
respondents may have occurred because SEAs were directly involved with contacting 
potential LEAs to complete the survey. This may have made local Special Education 
Directors feel obligated to complete the survey themselves. Additionally, in most states 
the survey was open during the summer months — a time when local Special Education 
Directors are usually more available than other staff. 

A third limitation was that two states limited their involvement to sharing information only 
on state level processes. No survey was conducted of LEAs within these two states. In 
both cases, personnel changes at the state level contributed to this decision. In light of 
the loss of two states, these findings do not reflect the entirety of Region 1 . 

Fourth, the timing of the survey administration study was difficult. The study was 
funded late in the school year. Creating the surveys took until the end of May for the 
first state and later for other states due to their workloads and internal schedules. 
Summer was a difficult time to get closure on the state-specific surveys. Repeated 
extensions of the survey deadline (through September) occurred in an effort to 
maximize the number of LEA responses. 

Finally, not surprisingly, states had different levels of interest and involvement. Some 
states got off to a fast start — others kept this project “on their desk” for many weeks 
before they began. One state had three involved people reviewing survey language, 
editing scenarios, and making recommendations for their state survey. Some states 
had to wait for approval from their SEA Director who was directly involved with 
reviewing. In other states, the Part B Data Manager was (by experience or default) 
empowered to make decisions and move the work forward. As such, the frequency, 
duration, and level of communication across states varied greatly. 

Policy Issue Results 

The purpose of this study was to collect data from states in the northeast and use the 
results to address three (original) specific LRE data policy questions and one additional 
technical assistance question. Findings relative to the policy questions are reviewed 
below. Policy questions 1 and 2 are combined due to their similar nature and the 
availability of data used to answer these two questions. 
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Policy Questions 1 and 2 

1. To what degree are state directions for reporting/counting children in specific 
placement categories consistent with OSEP directions? 

2. To what degree are state directions to LEAs for reporting/counting children in 
specific placement categories consistent across states? 

Question 1 inquires how each state’s directions differ from the OSEP directions. 
Question 2 inquires how state directions differ across northeast states. To answer 
these questions, this study looked at the LRE definitions each state used. The 
consultant asked each state to provide the exact LRE classification name and LRE 
definition each state used in 2005-06 for purposes of collecting and reporting 2005-06 
LRE data. Each state provided that information (or acknowledged that they used the 
OSEP definitions.). (See Table 6 for a summary of LRE categories by each state in the 
survey, and Appendix F for detail on state school age LRE definitions for all eight states 
in the northeast.) For the two states that did not participate in the survey (VT and Rl) 
LRE categories were gathered from state lEPs. 

Five of the eight states (CT, ME, NH, NY, and VT) used the exact OSEP 2005-06 LRE 
categories and definitions. The definitions used by the three remaining states (MA, NJ, 
Rl) are described below, with specific reference to how any changes might affect LRE 
data. 

Massachusetts 

Massachusetts uses a slightly different definition for the OSEP setting known as 
“Special Education outside the regular class less than 21 % of the day.” In 
Massachusetts, this setting is entitled: 

All ages, Full Inclusion: Special education services outside the General 
Education Classroom less than 21% of the time. 

The addition of the term “full inclusion” (and the additions of the terms “partial inclusion” 
and “substantially separate classroom” in the next two OSEP percent of day categories) 
may have the undesired consequence of confusing some staff that have the 
responsibility to classify and report LEA data. Hopefully, the vast majority of 
Massachusetts categorizers read the entirety of the definitions and understand the 
progression associated with time outside the general education classroom. However, 
the additional descriptors of “full inclusion” and “partial inclusion” could inadvertently 
cause some to misclassify certain students. This seems most possible when classifying 
those students receiving a small part (less than 21 % of the day) of their special 
education outside the regular class. Instead of being correctly placed in the 
Massachusetts category with the “full inclusion” descriptor, the unfamiliar might put such 
students in the category with the “partial inclusion” descriptor. This incorrect 
assumption could occur if they interpreted students receiving less than 100% of their 
special education outside the regular education classroom as being only partially 
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included and therefore placed them in the second percentage category with the “partial 
inclusion” label. 

Reviewing aggregate Massachusetts data substantiates that this topic may merit further 
investigation. If the term “full inclusion” would increase the likelihood that some 
students (who should correctly be classified in the “outside regular class less than 21% 
category”) are mistakenly placed in the 21 - 60% category, then Massachusetts’s data 
might show a proportionally lower percent of students in the “less than 21% category” 
than other states. Indeed, based on the 2004 data 1 (the latest data available when the 
proposal was written) at www.ideadata.org Massachusetts had the lowest percentage of 
students categorized “outside the regular class less than 21% of the day” of any 
northeastern state at 45.88%. Compared with their neighboring states, Massachusetts 
is 1 .25 standard deviations below the northeast mean (59.33%) for this category. Not 
unsurprisingly, their 2004 data were the highest (32.91 %) of any northeastern state for 
“outside regular class at least 21 percent of day and no more than 60 percent of day.” 

In this category — a category Massachusetts labels “partial inclusion” — Massachusetts is 
1.37 standard deviations above the regional norm. Overall, Massachusetts data and its 
comparison to other states within the region, suggests that this state’s placement 
definitions may be worthy of further exploration. 

Massachusetts includes an “All Ages” descriptor within their Part B LRE titles. Although 
the focus of this study is strictly school age Part B (not Preschool/619), this was 
deemed to have no affect on findings for this study. 

New Jersey 

New Jersey adopted and used in 2005-06 language that was proposed by OSEP for 
2006-07. Due to the consistency of the language in the OSEP definitions, this early 
adoption of OSEP language is not a definitional change or reinterpretation of existing 
categories. Conceptually, there exists a one-to-one relationship between the OSEP 
categories of 2005-06 and the OSEP categories of 2006-07. Because the New Jersey 
changes were clearly designed to emulate the future OSEP changes and these 
changes only affected three LRE categories (percentage of day calculations), this 
definitional shift is believed to have no affect on New Jersey’s year-to-year LRE data. 
Specific to this study, the language change had no affect and kept New Jersey’s data 
inline with the majority of other states. 2 

Rhode Island 

Rhode Island does not ask LEAs to categorize student LRE placement. Instead the 
SEA collects data on a series of questions. (See excerpt from Rl IEP in Appendix F.) 



1 During the course of this study the 2005 data were released. MA retained the highest percentage of all 
northeast states in the 21-60% category, with slight improvement in their under 21% category moving 
them up to an average of 49.27%, just under one standard deviation below the northeast region as a 
whole. 

2 Anytime change occurs — even simple language — there is potential likelihood of data shift. This can be 
due to changing databases, documents, applications, trainings, etc. 
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With these data, the SEA staff determines LRE categories by calculating the percentage 
of time in the regular class for each student. (Because their LEA constituency is not 
asked to categorize student LRE data, the survey portion of this study was not 
applicable in Rhode Island and was a large factor influencing their non-participation in 
the survey.) This study attempted to have those at the Rhode Island SEA who make 
the LRE placement determinations respond to the survey and/or provide a phone 
interview with the consultant on the Rhode Island process. Because Rhode Island did 
not choose to participate, it is unknown how the state’s LRE data categories classified 
at the SEA level affects their data. Reviewing the Rhode Island data for 2004 and 2005 
shows no particular anomalies. 

Policy Question 3 

3. How do LEAs’ understanding of those directions and the actual reporting of data 
compare or are comparable across LEAs within and across states? 

To truly answer policy question 3 a representative sample of LEAs would be needed 
from within each state. Since that was not possible within the limitations of this study, 
data were gathered that attempted to explore the questions surrounding LEA inter-judge 
reliability within and across northeast states. 

While a larger sample is needed to come to a definitive conclusion, the individual state 
graphs in Figure 2 suggest that LEA respondents in this study were not consistent 
within their states in classifying LRE placement scenarios. Except for the respondents 
from New York, in which a high degree of inter-rater reliability was demonstrated, 
respondents within each of the other states surveyed were never in total agreement 
regarding how to classify a single scenario. 

If respondents within a state are not in agreement in how to classify a scenario, it 
follows that little cross-state agreement would exist for the same scenario. The “all” 
state bar in the graphs in Figure 1 shows little agreement across states with respect to 
how respondents classified a scenario. There are, however, patterns of responses 
across states. This could mean that respondents in different states struggle similarly in 
their classification process. Although the study is based on a small and non- 
representative sample from the northeast, the findings indicate a need for cohesive and 
uniform training and technical assistance methods that cross state borders. Many of the 
recommendations below suggest regional and national (not just state) approaches for 
improving the reliability of the LRE classifications. 

In sum, generally states in the northeast (with the possible exception of New York) were 
not consistent in the way they individually classified the LRE placement of school age 
special education scenarios. While this study focused on the northeast states, there is 
no reason to suspect states in the northeast have any more or less internal LRE 
consistency than other regions of the U.S. 
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Policy Question 4 

In an effort to proactively provide additional information that might assist these SEAs 
this study took on a fourth policy question as part of this work. 

4. What types of technical assistance and training are available to LEAs on classifying 
LRE placement information and what is their level of utilization and perceived value? 

Tables 3 and 4 presented data from questions designed to elicit respondents’ a) 
knowledge of availability, b) engagement with, and c) perceived value of different types 
of SEA training and technical assistance. Results show that the respondents favored 
targeted technical assistance, that is, technical assistance designed to answer specific 
questions when needed. Data Dictionary, Expert Availability and Colleagues are 
examples of providing targeted (as opposed to general) assistance. 

The availability of a Data Dictionary is the most highly valued form of technical 
assistance, as reported by five respondents. (However, as reported in Appendix E, the 
existence of a Data Dictionary was never reported by a majority of respondents in any 
state, casting some doubt on respondent’s understanding of a Data Dictionary.) 
Generally, types of assistance characterized as “interactive” scored high. It seems 
important for respondents to communicate with others about a specific LRE placement 
issue — especially when they need immediate assistance. Trainings of a scheduled 
nature were noticeably less valued by respondents. It could be that such trainings are 
more general in nature — however the content of trainings was not part of this study. 

Some specific training improvements are included in the recommendations. With 
respect to the existence of training and technical assistance there was noticeable lack 
of knowledge among respondents about what training and TA is available within their 
state. (See Appendix E for more detailed data.) There was also considerable range in 
the degree of within state agreement regarding the availability of particular forms of 
assistance. The rate of agreement within states in terms of availability of particular 
forms of TA within the state ranged from a low of 18% in Massachusetts to a high of 
73% in New Jersey. This may in part be explained by the fact that the majority of 
respondents were local Special Education Directors — not necessarily the primary 
audience for trainings and ongoing TA on data collection. However, even local Special 
Education Directors should be expected to have an awareness of the types of SEA 
training and technical assistance available to their local staff. This is especially 
important when new personnel come into a position and may be inquiring of their 
Special Education Director about what available assistance. 

Training and technical assistance results influence the recommendations noted later in 
this section. It is hoped that the summary results — especially at the individual state 
level — will be useful to each state in this study. Logically reviewing training materials 
and processes would assist LEA staff in improving inter-rater reliability. 
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Suggestions for Future Studies 

The following suggestions are provided to assist future investigators of special 
education LRE classification. 

1 . Expand the sample. Any future study would do well to capture the input of many 
more respondents — ideally collect data on the universe of LEAs within a state. 

2. To the extent possible, use the same survey instrument. This will not only limit 
bias that can inadvertently be built into instruments by language choice, survey 
length, question order — but will simplify the development, streamline the 
communication, and reduce administration functions. 

3. If administering the survey via the web (recommended), program the survey 
software so that respondents that do not complete the survey are encouraged to 
do so before exiting. In this study, a small number of respondents started the 
survey but only answered a few questions before stopping. This made their 
results basically unusable. Given the limited number of respondents allowed per 
state, this was especially troublesome. 

4. Consider surveying multiple respondents within a district. For example, survey 
the LRE data knowledge of the Special Education Director and data secretary 
and a special education teacher. This would greatly increase the chance of 
accessing the person most likely to make LRE data classifications within that 
district. Second, it would provide insight into the LRE data knowledge of distinct 
groups of personnel employed in special education positions. 

5. Future investigations may wish to gather information on what would make 
selected trainings and technical assistance on LRE more or less useful. 

6. When possible, gather survey data from special education personnel during the 
school year, outside the months of June - August. 

7. Cleary communicate to State Directors of Special Education what commitments 
are necessary for study participation. Ask them to communicate these 
commitments to the IDEA Data Managers. Helping the SEA understand the 
state benefits of the study (e.g., target areas for training) may help ensure 
participation and responsiveness. 



State, Regional, and National Recommendations 

Aggregate LRE placement data are regularly reviewed and the results of LRE data 
collections can determine intervention at the state or federal level. These data are a 
measure for determining the degree to which students with disabilities have access to 
the general education curriculum and are receiving special education and related 
services in the least restrictive environment. Ideally, these data would be reliable and 
accurate and exhibit a very high degree of inter-rater agreement. Without acceptably 
high rates of inter-rater reliability, data are not comparable within or across states. 

Based on findings from this study, recommendations are made for improving LRE data 
reliability within LEAs, within a state, within a region, and therefore nationally. Within 
these recommendations “state” refers to State Departments of Education, “regional” 
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refers to Regional Resource Center (RRC) regions, and “national” refers to the 
assistance available via Westat / OSEP. 

1 . Discuss different ways scenarios are classified. (State, Regional, National) 

In this study many districts within a state classified a scenario differently from other 
districts within the same state. Frequent and open discussion — not lecture — that seeks 
to understand different interpretations can begin to inform documents and trainings that 
can help address state variation. To the extent this can be done regionally, or even 
nationally (through Data Manager meetings and discussion of common training 
materials) less variation will eventually be found in LRE data. Encouraging open 
discussion with peer-to-peer learning can move a state, region, and in time the country, 
towards greatly improved LRE inter-rater agreement. 

2. Update training materials (Regional, National) 

Develop regional and/or national training and TA materials specifically to assist SEAs in 
working with LEAs that, based on the data, may be experiencing difficulty in correctly 
coding LRE placements. Since states train on LRE in various ways, providing some 
somewhat standardized training materials for SEAs would be useful. Similarly to the 
present study, consider an investigation of how definitions, technical assistance, training 
and other factors might be affecting LRE data in states or regions. Then design 
materials accordingly. 

3. Create an extensive list of hard-to-code scenarios. (State, Regional, National) 

Collect the hardest-to-code real life scenarios at the local, state, regional, and national 
level. Gather and record all the facts available on the student in question. Change the 
details so it remains anonymous and determine the correct placement code. (This 
could be done by a well-informed and representative committee of experts.) Then build 
an online searchable and dynamic database where users can look up the scenarios on 
the Web. This would provide users at all levels with similar exemplars as well as an 
excellent teaching tool. 

4. Hotline for hard-to-code scenarios. (State, Regional, National) 

Respondents in this study reported high value to the availability of an expert being 
available via phone or email to assist with LRE coding. States might consider 
developing a customer hotline during the child count reporting period to immediately 
(within one working day) answer any LRE placement questions. Ideally, Westat / OSEP 
could provide this same level of timely response when SEA Data Managers have similar 
questions. State IDEA Data Managers need to be seen as responsive to their LEA 
constituents and the “teachable moment” is immediately after a question is asked. To 
the extent that the most difficult to code scenarios could be pre-captured and classified 
in state, regional, or national training materials (#2 above), an immediate national 
hotline would be less needed. 

5. Improve state level trainings. (State, Regional, National) 

Based on this study, state level trainings to LEAs of a face-to-face and hands-on nature 
are frequent but not as highly valued as other methods of knowledge transfer. Consider 
what needs to be done to improve such trainings, such as: 
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■ Having attendees bring one or two examples of difficult to code students 
based on real life. Discuss the scenarios as a group. Share materials (SEA, 
RRC, National) that are designed to assist the LEAs in classifying LRE 
scenarios. 

■ Assist LEAs in reviewing their year-to-year change reports (if available). 
Alternatively, have LEAs pull and analyze their data. 

■ Survey participant satisfaction with SEA trainings (and optionally, other 
methods of technical assistance). Based on responses, incorporate new 
methods and materials into trainings. 

■ Consider holding a focus group specifically for soliciting feedback on data 
trainings. Concentrate on how to improve all aspects of LRE training and 
technical assistance. 

6 . Consider target audience to announce trainings. (State) 

The respondents for this study were predominantly local Special Education 
Directors — and their self-report data suggests they were not well informed about the 
existence of specific LRE technical assistance and trainings. Depending on 
communication policies, copy local directors on all e-mail and paper correspondence 
relative to LRE assists. Also at SEA conferences consider having a data support staff 
at a table for a day to share with interested local special education directors what data 
collections exist, what trainings exist, due dates, etc. Providing an attractive one page 
annual calendar of data events is particularly helpful to visual learners. Encourage local 
Special Education Directors to contact the SEA whenever a new local data support 
person is hired. 

7. Develop LEA year-to-year change reports. (State, Regional) 

Similar to Westat’s use of SEA level data, SEAs should develop reports that present to 
the LEA both in graphic and tabular form the LRE changes within a district from one 
year to the next. Ideally, these can be viewed by the LEA while they are in the last 
stages of their current year submission process. LEAs with changes that exceed a 
designated level should be required to explain the changes to the SEA. (Occasionally, 
data system errors can account for substantial shifts in an LEA’s data. Providing year- 
to-year change reports assists in providing an early alert to the LEA.) The Regional 
Resource Centers could assist one or more states that are not doing this by 
coordinating collaborative sharing within the region, and across regions, helping SEAs 
develop year-to-year report formats, and coordinating the development of LEA training 
materials for using the reports. 

8 . Develop student-level year-to-year change reports. (State, Regional) 

States with individual student level collection capacity can design student specific 
reports to flag whenever an individual student moves two or more points on the LRE 
placement continuum from one year to the next. LEAs with such students could then be 
required to provide an explanation whenever a preset number and/or percent of such 
changes exist. Such data could also be useful for LEA monitoring. State assigned 
unique identifiers are required. The Regional Resource Center could assist states in 
the region to design such reports and LEA training materials. 
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9. Develop a pre-test for IDEA Data Managers. (State) An additional idea, not 
based on any specific aspect of this study, is to develop a pre-test for IDEA Data 
Managers. A pre-test would serve as an introduction to the complexity of issues 
surrounding LRE. It might encourage new (and experienced) IDEA Data Managers to 
access the resources and technical assistance available on LRE. A Regional Resource 
Center could also develop such a test for fostering regional discussions with state IDEA 
Data Managers. 

Conclusion 

This study began with the premise that LRE data are important; therefore accuracy 
(reliability) of LRE data is critical. Based on a small number of respondents, this study 
sheds doubt about the accuracy of LRE data within and across the northeastern states. 
However, there is no reason to believe that this group of states is any more or less 
different than any other group of states. It is our belief that many, possibly most, SEA 
IDEA Data Managers suspect inter-rater reliability problems in the collection of LRE 
data within their own state. Findings and recommendations from this study can be used 
to improve the accuracy of LRE data at classroom, district, state, region and national 
levels. 



The study was conducted under the direction of the Northeast Regional Resource Center 
(NERRC) Director Kristin Reedy, Ed.D. Bruce Bull of Education Data Services and Information 
Systems (EDSIS) was the consultant and conducted the work. 
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Appendix A 

Northeast States LRE Data (Based on 2004 Data) 

Students placed outside the regular classroom for less than 21% of the time: States in 
Region 1 demonstrate a range from a low of 44.23% in Massachusetts to a high of 76.54% in 
VT. Six out of eight states are above the U.S. average of 52.14%. This is an indication that the 
region is highly “inclusive” in terms of the numbers of students that receive the majority of their 
educational program in the regular classroom. 

Special education outside regular class at least 21 percent of day and no more than 60 
percent of the time: Region 1 states demonstrated a range from a low of 12.04% in NY to a high 
of 32.91% in Massachusetts. 

Students placed outside the regular classroom for greater than 60% of the time: States in 
Region 1 demonstrate a range from a low of 3.32% in NH to a high of 27.32% in NY. Only two 
of the eight states exceed the U.S. average of 17.51% (NY and RI). 

Students in separate facilities: For the 2000-2001 school year, the last year that state rank- 
ordered data were published by OSEP, states in Region 1 demonstrated a range from a low of 
1.3% in ME to a high of 9.1% in NJ. Six of the eight states exceeded the U.S. average of 3.0%. 
Given the relatively high rates of placement in the regular classroom for the majority of the 
school day, it is interesting to note the high rates of placement in separate facilities demonstrated 
across the region. In fact, on OSEP’s rank ordered list of states (2000-01), six of the eight 
northeastern states fell within the bottom quarter of states in the U.S. For 2004, states in Region 
1 demonstrate a range from a low of .10% in NH for public separate facilities to a high of 
3.36% in NJ. Four out of eight states are above the U.S. average of 1.77% in this placement 
category. For private separate facilities, the range is from 1.76% in ME to 5.57% in NJ. All eight 
Region 1 states exceed the U.S. average of 1.20% for private separate facilities. 



Table A-1. Percent of students ages 6 through 21 served under IDEA by educational environment and state 



Percent of students in all environments 
2004 Data 

- Outside regular class - 



2004 Data 
State 


<21% 

(%) 


21-60% 

(%) 


> 60% 
(%) 


Public 

separ 

facil 

(%) 


Private 

separ 

facil 

(%) 


Public 

resid 

facil 

(%) 


Private 

resid 

facil 

(%) 


Home 

hosp 

envir 

(%) 


All 

envir 

(%) 


Connecticut 


60.67 


22.88 


10.30 


1.62 


2.84 


0.11 


1.42 


0.16 


100.00 


Maine 


55.38 


29.20 


11.69 


0.96 


1.76 


0.03 


0.59 


0.41 


100.02 


Massachusetts 


44.23 


32.91 


16.11 


2.48 


3.13 


0.11 


0.88 


0.15 


100.00 


New Hampshire 


75.52 


16.81 


3.32 


0.10 


2.54 


0.20 


1.31 


0.19 


99.99 


New Jersey 


45.88 


27.85 


16.65 


3.36 


5.57 


0.16 


0.10 


0.44 


100.01 


New York 


53.64 


12.04 


27.32 


2.85 


2.46 


0.38 


0.84 


0.47 


100.00 


Rhode Island 


62.76 


13.84 


18.70 




3.09 




0.88 


0.50 


99.77 


Vermont 


76.54 


7.88 


8.74 


1.95 


3.30 


0.09 


1.20 


0.31 


100.01 
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NE Mean 
Standard Deviation 


59.33 

12.12 


20.43 

9.09 


14.10 

7.29 


1.90 

1.12 


3.09 

1.12 


0.15 

0.11 


0.90 

0.43 


0.33 

0.15 




2005 Data 


State 


Outside regular class 

<21% 21-60% >60% 

(%) (%) (%) 


Public 

separ 

facil 

(%) 


Private 

separ 

facil 

(%) 


Public 

resid 

facil 

(%) 


Private 

resid 

facil 

(%) 


Home 

hosp 

envir 

(%) 


All 

envir 

(%) 


Vermont 


77.80 


7.69 


8.56 


0.56 


4.11 




1.08 




100.00 


Rhode Island 


64.30 


12.94 


18.27 




3.11 




0.83 


0.31 


100.00 


New York 


54.51 


13.06 


25.51 


2.72 


2.53 


0.39 


0.75 


0.52 


100.00 


New Hampshire 


76.26 


16.17 


3.23 


0.07 


2.64 


0.17 


1.31 


0.16 


100.00 


Connecticut 


65.21 


20.40 


7.66 


1.78 


3.32 


0.09 


1.37 


0.17 


100.00 


New Jersey 


46.05 


27.95 


16.68 


3.16 


5.44 


0.15 


0.08 


0.50 


100.00 


Maine 


57.11 


28.19 


11.17 


0.83 


1.75 


0.04 


0.53 


0.38 


100.00 


Massachusetts 


49.27 


28.45 


15.60 


2.45 


3.14 


0.12 


0.85 


0.12 


100.00 


NE Mean 


61.31 


19.36 


13.34 


1.65 


3.26 


0.16 


0.85 


0.31 


100.00 


Standard Deviation 


11.72 


8.13 


7.08 


1.19 


1.12 


0.12 


0.42 


0.16 
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Appendix B 

Advance Project Checklist for LRE Materials 



State: Date: 



LRE-Specific Material 


Available 
to Review 


Not 

Available 


Not 

Sure 


On 

Web? 


Comment 


1. Copies of Forms 












2. State Data Dictionary 
with LRE Definitions 












3. Online Help from State 
Data System(s) 












4. Face-to-Face Training 
Materials 












5. Training Videos / 
Online Flash 












6. FAQs Regarding LRE 












7. Other 












8. Other 












9. Other 












Initial Thoughts on 
Expanded Survey Option 




General Comments 





State Materials Form 



3S 
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Appendix C 
Process Form 

A portion of this LRE study looks at processes that can affect the way LRE data are reported. 
Please answer each question and submit completed questionnaires at the end of today’s 
lunch meeting. Thank you! 

1 . Briefly, describe your SEA’s internal process to implement any annual OSEP changes to 
IDEA data requirements and/or definitions? (E.g., informal discussions, formal 
committee(s) review, IT submission process.) 

2. If/when your SEA can not/will not abide by a new data requirement or definition (for the 
immediate future or long term), what internal SEA processes is there for making this 
decision? 

For the remainder of the items please think specifically about LRE. 

3. Does your state require LRE definitions on the IEP? If yes, to what degree do the LRE 
definitions exactly mirror OSEP definitions? 

4. What materials does your SEA use to inform LEAs of updates to LRE definitions? 

5. What processes does your SEA use to inform LEAs of updates to LRE definitions? 

6. Indicate the feedback your SEA provides to LEAs relative to their submitted LRE data. 

a) Are year-to-year changes in annual child count LRE data brought to the LEA’s 
attention? If so, when? 

b) Are LRE trend analyses conducted by the SEA and reported to the LEA? 

c) Are LRE trend analyses conducted by the LEA that is monitored or reviewed by the 
SEA? 

d) Are student level change reports provided back to the LEA by the SEA? 

e) Does the state use a verification process (e.g., sampling of files) to check reported 
LRE data at the individual student level? If yes, describe. 

f) Other? 

7. How do LEAs communicate the definitions (old or updated) to those responsible for 
transmitting LRE data from the IEP to the SEA? 

8. Finally, to what degree are the OSEP changes received by your SEA in time to make the 
necessary changes in the collection processes 
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Appendix D 

Generic Survey Instrument 

Actual surveys were state specific and built in a program called SelectSurvey. Below is 
the text of all the questions that were asked of all states. 

Introduction 

This survey is designed to gather information on how the educational placement of students with 
disabilities is determined and reported to the State Education Agency (SEA). The Northeast Regional 
Resource Center (NERRC) received a grant from Westat to conduct a regional study of the special 
education placement patterns and processes in the northeastern states. This survey is part of that work. 
Your responses can remain anonymous — no names are requested. Your responses will be combined 
with those from other Local Education Agencies (LEAs) in your state and in the northeast region, to 
provide insight on the topic of placement. 

Access to student records is not required for this survey, and the survey can be completed from any 
location with Internet access. This is NOT a state monitoring activity. It is NOT part of any LEA 
investigation. And it is NOT part of any child count data verification process. It is only a state affiliated 
activity in that SEA directors from all of the northeastern states supported NERRC in proposing and 
conducting this study. 

Challenges interacting with the online survey process should be addressed to Bruce Bull. (503.589.9660 
x 21, bruce.bull@edsis.org.) He is in the Pacific time zone and therefore 3 hours behind us. Questions 
regarding survey content or a specific item should be addressed to: or @- 



We truly appreciate your time (estimated survey response time 15-20 minutes) and input on this topic. 

Demographics / Experience 

1 . Pick the one category that most accurately categorizes your current job: * 

_ Special Ed Secretary/Administrative Assistant /Child Count Data Entry Clerk 
_ IEP Team Chair / Special Ed Liaison 
_ Special Education Teacher 

_ Special Education Director or Director of Special Education 
_ Other, please specify 

2. How many YEARS have you been in the above position? (Round to nearest whole year.)* The value 
must be between 0 and 40, inclusive. 

3. In general, what is your level of confidence in the accuracy of the data you are reporting to your SEA 
on the placement classifications of students with disabilities? * 

1 (Low) 2 3 4 5 (High) Confidence 

Technical Assistance and Training 

We want to know about the availability and usefulness of different types of technical assistance (TA) 
and/or training you may have access to and/or received specifically on the topic of categorizing and 
reporting placement data to the SEA. 

4. Were FACE-TO-FACE trainings/workshops available regarding placement data? * Yes No 

5. In the last 24 months, did you engage in any FACE-TO-FACE trainings/workshops regarding 
placement data?* Yes No 

6. How valuable to you was that FACE-TO-FACE training/workshop?* 

1 (Low) 2 3 4 5 (High) Value 
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7. Were DISTANCE LEARNING trainings/workshops available regarding placement data? * 

Yes No 

8. In the last 24 months, did you engage in any DISTANCE LEARNING trainings/workshops regarding 
placement data?* Yes No 

9. How valuable to you was that DISTANCE LEARNING training/workshop?* 

1 (Low) 2 3 4 5 (High) Value 

10. Were HANDS-ON computer trainings/workshops available regarding placement data? (These may 
have been in conjunction with face-to-face trainings or distance learning trainings.) * Yes No 

11. In the last 24 months, did you engage in any HANDS-ON trainings/workshops regarding placement 
data?* Yes No 

12. How valuable to you was that HANDS-ON training/workshop?* 1 (Low) 2 3 4 5 (High) Value 

13. Was VIDEO of taped trainings (e.g., VHS, online) available regarding placement data? * 

Yes No 

14. In the last 24 months, did you view any VIDEO of taped trainings/workshops regarding placement 
data?* Yes No 

1 5. How valuable to you was that VIDEO of trainings/workshops regarding placement data?* 

1 (Low) 2 3 4 5 (High) Value 

16. Was AUTOMATED HELP available from the data system or application (e.g., online help screens)? * 
Yes No 

17. In the last 24 months, did you engage any AUTOMATED HELPS from the data system or 
application? * Yes No 

18. How valuable to you was that AUTOMATED HELP?* 1 (Low) 2 3 4 5 (High) Value 

19. Was a DATA DICTIONARY available to you regarding placement data? * Yes No 

20. In the last 24 months, did you access the DATA DICTIONARY regarding placement data?* 

Yes No 

21. How valuable to you was that DATA DICTIONARY?* 1 (Low) 2 3 4 5 (High) Value 

22. Were DOCUMENTS such as memos, letters, process manuals, etc. (other than a Data Dictionary) 
available to help you address the topic of education placement? * Yes No 

23. In the last 24 months, did you access any DOCUMENTS (other than a Data Dictionary) to assist you 
with placement data?* Yes No 

24. How valuable to you were those DOCUMENTS (other than a Data Dictionary)?* 

1 (Low) 2 3 4 5 (High) Value 

25. Was a "state EXPERT" available to you via EMAIL to answer questions you had regarding education 
placement?* Yes No 

26. In the last 24 months, did you engage a state "EXPERT" via EMAIL to assist you regarding 
placement data?* Yes No 

27. How valuable to you was the EMAIL assistance from the state "EXPERT"?* 
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1 (Low) 2 3 4 5 (High) Value 

28. Was a "state EXPERT" available to you via PHONE to answer questions you had regarding 
education placement?* Yes No 

29. In the last 24 months, did you engage a state "EXPERT" via PHONE to assist you regarding 
placement data?* Yes No 

30. How valuable to you was the PHONE assistance from the state "EXPERT"?* 

1 (Low) 2 3 4 5 (High) Value 

31. Were FAQs (frequently asked questions with answers) available to you regarding placement data? 
Yes No 

32. In the last 24 months, did you access the FAQs regarding placement data?* Yes No 

33. How valuable to you were these FAQs?* 1 (Low) 2 3 4 5 (High) Value 

34. Was HELP FROM A COLLEAGUE available to you regarding placement data? * Yes No 

35. In the last 24 months, did you access any HELP FROM A COLLEAGUE regarding placement data?* 
Yes No 

36. How valuable to you was that HELP FROM A COLLEAGUE?* 1 (Low) 2 3 4 5 (High) Value 



37. Please share any other types of TA you received or had available to you regarding understanding 
and reporting special education placement data. You may also provide additional comments on any of the 
above types of technical assistance if you wish. 



38. When you have a student that is difficult to code, what process(es) do you use to determine the 
placement? * 



a) Take my best educated guess 

b) Ask a colleague 

c) Call someone at the state 

d) Review TA materials 



Never Rarely 
Never Rarely 
Never Rarely 
Never Rarely 



Sometimes 

Sometimes 

Sometimes 

Sometimes 



Often Always 
Often Always 
Often Always 
Often Always 



39. What else would assist you in classifying special education placement determinations? 



Scenarios 

As you know, a student may receive education in multiple settings (e.g., regular class, self contained 
special education class, private residential school). At times, categorizing educational placement can be 
difficult. This survey is designed to get information on “difficult to code” school age students. Getting your 
honest and independent perspective will provide us insight into placement categorization. 

Please read each school age scenario that follows and pick the best response base on the "School Age 
Definitions". (School age definitions are contained in the link.) 

On some scenarios you may wish you had additional information to categorize more accurately. However, 
there is no additional information. As you may suspect, we have created some scenarios that are difficult 
to code. Also, this is not a “group exercise.” Getting your independent perspective is important. 
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40. Steve’s parents removed Steve, a 7th grader, from public school and placed him in a private prep 
school in their district. Later Steve is found eligible for special education. Steve now receives 2 hours a 
day of pull-out services in the resource room at the private school. * 

_ Sp Ed outside regular class less than 21 % of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
_ Don't know 

41. Jamie goes to the local high school in the morning for 15 hours a week. While there Jamie receives 5 
hours a week of special education in a co-taught program. (The co-taught classroom has one special 
education teacher and one general education teacher. The general education teacher is the teacher of 
record for this classroom.) In the afternoon Jamie goes to a work study program at the local McDonald's 
for a total of 15 hours a week.* 

_ Sp Ed outside regular class less than 21 % of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
_ Don't know 

42. A district places Holly at a residential school for students with sensory impairments. Holly lives close 
to the school, so she takes the bus to and from school as a day student. Assume this is a public 
residential school.* 

_ Sp Ed outside regular class less than 21 % of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
_ Don't know 

43. What is Holly's placement if the school is a private residential school?* 

_ Sp Ed outside regular class less than 21 % of day 

_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
Don't know 



44. The district placed Pat at a private separate school for students on lEPs. Though still attending there 
part of the day, he is now transitioning back to his local high school. Currently his daily routine is to attend 
the private school from 9 - 12 then he is bussed to the public high school. In the afternoon he is in two 
general education classes with an instructional assistant who is responsible for Pat and another student 



Northeast. Rpfrintuil Rpanurcp C.pnlpv 



40 




Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



with disabilities. He also attends one special education life skills class at the public high school. Each high 
school class is an hour long.* 

_ Sp Ed outside regular class less than 21 % of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
_ Don't know 

45. Jane is a parentally placed private school student with a Service Plan. During the first part of the year 
she received 1 hour of speech/week on a “pull-out” basis. The district decided to put her on an IEP in 
November but service levels and placement remain the same. * 

_ Sp Ed outside regular class less than 21 % of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
_ Don't know 

46. Matt has two classes in community job experience, two general education content courses with 
special education support as needed, one class in a resource room with a special education teacher on 
study skills, and one elective class with no additional support. He has no other classes and the frequency 
and duration of all classes is the same. One of the community job experience classes is a general ed 
class, the other is a special education class. Also, in one general education class Matt is counted on 
enrollment by the special education teacher, in the other class the regular ed teacher counts him for 
attendance. * 

_ Sp Ed outside regular class less than 21% of day 
_ Sp Ed outside reg cl at least 21% not more than 60% 

_ Sp Ed outside regular class more than 60% of day 
_ Public separate school 
_ Private separate school 
_ Public residential facility 
_ Private residential facility 
_ Homebound/Hospital 
Don't know 
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Appendix E 

Table E-1. Summary results of availability, engagement, and value of different technical 
assistance and training methods 





CT 


MA 


ME 


NH 


NJ 


NY 


Sums 


Score if 
yes 


FACE-TO-FACE available 


9 of 9 


6 of 8 


3 of 6 


5 of 8 


4 of 6 


6 of 8 


33 




FACE-TO-FACE engage 


6 of 9 


4 of 6 


2 of 3 


3 of 5 


3 of 4 


5 of 6 


23 




FACE-TO-FACE value 


4.50 


3.75 


3.00 


3.00 


4.00 


3.80 


88.00 


3.83 


DISTANCE LEARNING available 


Oof 9 


1 of 8 


3 of 6 


1 of 8 


1 of 6 


1 of 8 


7 




DISTANCE LEARNING engage 




Oof 1 


1 of 3 


0 if 1 


1 of 1 


Oof 1 


2 




DISTANCE LEARNING value 






1.00 




5.00 




6.00 


3.00 


HANDS ON available 


2 of 9 


4 of 8 


2 of 6 


4 of 8 


2 of 6 


2 of 8 


16 




HANDS ON engage 


1 of 2 


3 of 4 


2 of 2 


3 of 4 


2 of 2 


1 of 2 


12 




HANDS ON value 


5.00 


4.33 


3.00 


2.33 


4.00 


5.00 


44.00 


3.67 


VIDEO available 


2 of 9 


2 of 8 


1 of 6 


0 of 8 


Oof 6 


0 of 8 


5 




VIDEO engage 


0 of 2 


1 of 2 


Oof 1 








1 




VIDEO value 




4.00 










4.00 


4.00 


AUTOMATED HELP available 


2 of 9 


3 of 8 


3 of 5 


1 of 8 


1 of 6 


4 of 8 


14 




AUTOMATED HELP engage 


1 of 2 


1 of 3 


3 of 3 


Oof 1 


Oof 1 


2 of 4 


7 




AUTOMATED HELP value 


4.00 


4.00 


3.33 






3.00 


24.00 


3.43 


DATA DICTIONARY available 


2 of 9 


1 of 8 


0 of 4 


4 of 8 


2 of 6 


4 of 8 


13 




DATA DICTIONARY engage 


1 of 2 


1 of 1 




1 of 4 


0 of 2 


2 of 4 


5 




DATA DICTIONARY value 


5.00 


5.00 




4.00 




5.00 


24.00 


4.80 


DOCUMENTS available 


9 of 9 


4 of 8 


2 of 4 


7 of 8 


6 of 6 


8 of 8 


36 




DOCUMENTS engage 


7 of 9 


4 of 4 


2 of 2 


4 of 7 


2 of 6 


8 of 8 


27 




DOCUMENTS value 


4.29 


4.00 


3.50 


3.00 


3.50 


4.00 


104.00 


3.85 


EXPERT EMAIL available 


9 of 9 


5 of 8 


3 of 4 


7 of 8 


5 of 6 


5 of 8 


34 




EXPERT EMAIL engage 


8 of 9 


4 of 5 


3 of 3 


5 of 7 


3 of 5 


3 of 5 


26 




EXPERT EMAIL value 


4.88 


5.00 


3.00 


3.40 


4.67 


4.00 


111.00 


4.27 


EXPERT PHONE available 


9 of 9 


5 of 8 


3 of 4 


7 of 8 


6 of 6 


7 of 8 


37 




EXPERT PHONE engage 


8 of 9 


3 of 5 


3 of 3 


5 of 7 


4 of 6 


5 of 7 


28 




EXPERT PHONE value 


4.88 




3.67 


3.20 


5.00 


4.40 


123.00 


4.39 


FAQ available 


5 of 9 


5 of 8 


1 of 4 


2 of 8 


5 of 6 


5 of 8 


23 




FAQ engage 


4 of 5 


5 of 5 


Oof 1 


1 of 2 


4 of 5 


5 of 5 


19 




FAQ value 


4.00 






4.00 


3.75 


4.00 


76.00 


4.00 


COLLEAGUE available 


9 of 9 


5 of 8 


2 of 4 


7 of 8 


6 of 6 


7 of 8 


36 




COLLEAGUE engage 


8 of 9 


3 of 5 


2 of 2 


7 of 7 


4 of 6 


6 of 7 


30 




COLLEAGUE value 


3.88 


4.33 


4.00 


4.29 


4.75 


4.50 


128.00 


4.27 


# consistent / possible 


6/11 


2/11 


5/11 


7/11 


8/11 


5/11 




33/66 






= Consistent 
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Appendix F 

State School Age LRE Definitions 
Rhode Island 

(Information secured from IEP document linked from Rl DOE.) 



31) SPECIAL EDUCATION AND RELATED SERVICES 


32) 

GOAL# 


33) PROVIDERS/ 
SERVICES 


34) LOCATION/ 
PLACEMENT 


35) FREQUENCY/ 
DURATION 


36) DATE 


37) EXPLANATION 
FOR PLACEMENT 
OUTSIDE OF 
GENERAL 
EDUCATION 
SETTING 


REG ED 


OTHER 


HRS / 
DAY 


DAYS 

/WK 


WK/ 

MO 


FROM 


TO 
























38) DETE 

a) 1 

b) 1 

c) 1 


RMINATION OF TIME IN REGI 

r otal time in student’s school w< 

r otal time of Special Education 
'ime in regular education withoi 


JLAR EDI 

sek 

Support/S 
jt Special 


JCATION PROGRAM 

ervices (33) 

Education Support/Se 


rvices 


39) EXTENT OF PARTICIPATION IN REGULAR EDUCATION: This student will participate in regular grade 
level program (for middle/junior high school/high school, attach schedule) except 



Vermont 

(Information secured from state IEP form as Vermont was not involved with survey. 

Information is reformatted here.) 



The general characteristics of the student/child’s placement (check one, ages 6-21): 



Inside regular class at least 80% of the time 
Inside regular class 40% to 79% of the time 
Inside regular class less than 40% of the time 
Separate day school - public or private 

Residential facility 

Homebound/Hospital 



Northeast Regional Resource Center 



47 



Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



New Jersey 

Educational Environments of Children with Disabilities Ages 6-21 (NJ) 

To calculate the percentage of time inside the regular classroom, divide the number of hours the youth 
receives special education and related services inside the regular classroom by the total number of hours 
in the school day (including lunch, recess and study periods). 

Home-schooled students who receive special education and related services provided by the local 
educational agency should be counted in the first three listed placements below. For these students, the 
number of hours outside the regular classroom is the number of hours the youth spends in separate 
special education environments (e.g. resource rooms, self-contained special education classrooms, 
separate schools). 

Educational time spent in age-appropriate community-based settings that include individuals with and 
without disabilities, such as college campuses or vocational sites, should be counted as time spent with 
non-disabled peers. 

Students with disabilities who are included in the general education setting at least 80% of the 
school day. This may include children with disabilities placed in: 

• regular class with special education/related services provided within regular classes; 

• regular class with special education/related services provided outside regular classes; or 

• regular class with special education services provided in resource rooms. 

Students with disabilities in the presence of general education students between 40% and 80% of 
the entire school day. Students with disabilities who are included in the general education setting 
from 40% to 79% of the school day. Includes children with disabilities receiving special education and 
related services inside the regular classroom for at least 40% but no more than 79% of the school day. 

No more than 60% of their special education and related services are provided outside the general 
education classroom. Do not include children who are reported as receiving education programs in public 
or private separate school or residential facilities. This may include children placed in: 

• resource rooms with special education/related services provided within the resource room; or 

• resource rooms with part-time instruction in a regular class. 

Students with disabilities who are included in the general education setting up to 39% of the 
school day. At least 61% of their special education and related services are provided outside the 
general education classroom. Includes children with disabilities receiving special education and related 
services inside the regular classroom for less than 40 percent of the school day. Do not include children 
who are reported as receiving education programs in public or private separate school or residential 
facilities. This category may include children placed in: 

• self-contained special classrooms with part-time instruction in a regular class; or 

• self-contained special classrooms with full-time special education instruction on a regular 
school campus. 

Public separate school. Students with disabilities who receive education programs in public separate 
day school facilities. This includes children with disabilities receiving special education and related 
services for greater than 50 percent of the school day in public separate schools. This may include 
children placed in: 

• public day schools for students with disabilities; 

• public day schools for students with disabilities for a portion of the school day (greater than 
50 percent) and in regular school buildings for the remainder of the school day; or 

• public residential facilities if the student does not Ijve at the facility. 

Private separate school. Students with disabilities who receive education programs in private separate 
day school facilities. This includes children with disabilities receiving special education and related 
services, at public expense, for greater than 50 percent of the school day in private separate schools. 

This may include children placed in: 
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• private day schools for students with disabilities; 

• private day schools for students with disabilities for a portion of the school day (greater than 

50 percent) and in regular school buildings for the remainder of the school day; or 

• private residential facilities if the student does not live at the facility. 

Public residential facility. Students with disabilities who receive education programs and live in public 
residential facilities during the school week. This includes children with disabilities receiving special 
education and related services for greater than 50 percent of the school day in public residential facilities. 
This may include children placed in: 

• public residential schools for students with disabilities; or 

• public residential schools for students with disabilities for a portion of the school day (greater 

than 50 percent) and in separate day schools or regular school buildings for the remainder of 
the school day. 

Do not include students who received education programs at the facility, but do not live there. 

Private residential facility. Students with disabilities who receive education programs and live in private 
residential facilities during the school week. This includes children with disabilities receiving special 
education and related services, at public expense, for greater than 50 percent of the school day in private 
residential facilities. This may include children placed in: 

• private residential schools for students with disabilities; or 

• private residential schools for students with disabilities for a portion of the school day (greater 
than 50 percent) and in separate day schools or regular school buildings for the remainder of 
the school day. 

Do not include students who received education programs at the facility, but do not live there. 

Homebound/Hospital. Students with disabilities who receive education programs in 
homebound/hospital environment includes children with disabilities placed in and receiving special 
education and related services in: 

• hospital programs, or 

• homebound programs. 

Do not include children with disabilities whose parents have opted to home-school them and who receive 
special education at the public expense. 



Massachusetts 



DOE034 

Educational Environment 



3-5 Year Olds ONLY: 

08- 100% services in general education classroom 

09- 100% services in separate classroom 

ALL AGES: 

10- Full Inclusion, less than 21% outside general education 
20- Partial Inclusion, 21%-60% outside general education 

40- Substantially Separate, outside general education more than 60% 

41 - Public Separate Day 
50-Private Separate Day 
60-Residential School 

70-Homebound/Hospital (Not home schooled) 

90-Public Residential Facility 



Northeast Repional Resource Center 



45 



Region 1 2006 Westat State Analysis LRE Grant: Final Report 



January 9, 2007 



New Hampshire, New York, Connecticut, Maine 

Educational Environments of Children with Disabilities Ages 6-21 

NY To calculate the percentage of time outside the regular classroom, divide the number of hours the 
youth receives special education and related services outside the regular classroom by the total number 
of hours in the school day (including lunch, recess and study periods). 

ME Home-schooled students who receive special education and related services provided by the local 
educational agency should be counted in the first three listed placements below. For these students, the 
number of hours outside the regular classroom is the number of hours the youth spends in separate 
special education environments (e.g. resource rooms, self-contained special education classrooms, 
separate schools). 

ME Educational time spent in age-appropriate community-based settings that include individuals with and 
without disabilities, such as college campuses or vocational sites, should be counted as time spent with 
non-disabled peers. 

Special education outside regular class less than 21 percent of day. Unduplicated number of 
children with disabilities receiving special education and related services outside the regular classroom 
for less than 21 percent of the school day. This may include children with disabilities placed in: 

• regular class with special education/related services provided within regular classes; 

• regular class with special education/related services provided outside regular classes; or 

• regular class with special education services provided in resource rooms. 

Special education outside regular class at least 21 percent of day and no more than 60 percent of 
day. Unduplicated total includes children with disabilities receiving special education and related services 
outside the regular classroom for at least 21 percent but no more than 60 percent of the school day. Do 
not include children who are reported as receiving education programs in public or private separate 
school or residential facilities. This may include children placed in: 

• resource rooms with special education/related services provided within the resource room; or 

• resource rooms with part-time instruction in a regular class. 

Special education outside regular class more than 60 percent of day. Unduplicated total includes 
children with disabilities receiving special education and related services outside the regular classroom 
for more than 60 percent of the school day. Do not include children who are reported as receiving 
education programs in public or private separate school or residential facilities. This category may include 
children placed in: 

• self-contained special classrooms with part-time instruction in a regular class; or 

• self-contained special classrooms with full-time special education instruction on a regular 
school campus. 

Public separate school. Unduplicated total who received education programs in public separate day 
school facilities. This includes children with disabilities receiving special education and related services 
for greater than 50 percent of the school day in public separate schools. This may include children 
placed in: 

• public day schools for students with disabilities; 

• public day schools for students with disabilities for a portion of the school day (greater than 
50 percent) and in regular school buildings for the remainder of the school day; or 

• public residential facilities if the student does not Ijve at the facility. 

Private separate school. Unduplicated total who received education programs in private separate day 
school facilities. This includes children with disabilities receiving special education and related services, 
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at public expense, for greater than 50 percent of the school day in private separate schools. This may 
include children placed in: 

• private day schools for students with disabilities; 

• private day schools for students with disabilities for a portion of the school day (greater than 

50 percent) and in regular school buildings for the remainder of the school day; or 

• private residential facilities if the student does not Ijve at the facility. 

Public residential facility. Unduplicated iota\ who received education programs and lived in public 
residential facilities during the school week. This includes children with disabilities receiving special 
education and related services for greater than 50 percent of the school day in public residential facilities. 
This may include children placed in: 

• public residential schools for students with disabilities; or 

• public residential schools for students with disabilities for a portion of the school day (greater 

than 50 percent) and in separate day schools or regular school buildings for the remainder of 
the school day. 

Do not include students who received education programs at the facility, but do not live there. 

Private residential facility. Unduplicated total who received education programs and lived in private 
residential facilities during the school week. This includes children with disabilities receiving special 
education and related services, at public expense, for greater than 50 percent of the school day in private 
residential facilities. This may include children placed in: 

• private residential schools for students with disabilities; or 

• private residential schools for students with disabilities for a portion of the school day (greater 
than 50 percent) and in separate day schools or regular school buildings for the remainder of 
the school day. 

Do not include students who received education programs at the facility, but do not live there. 

Homebound/Hospital. Unduplicated total who received education programs in homebound/hospital 
environment includes children with disabilities placed in and receiving special education and related 
services in: 

• hospital programs, or 

• homebound programs. 

Do not include children with disabilities whose parents have opted to home-school them and who receive 
special education at the public expense. 
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Appendix G 

Aggregate and State Level Consensus on Coding LRE for School Age Scenarios 

How to read this table: 

The table below provides the scenarios followed by a heading row with headings for 
aggregate data (AG1 and AG2) and state data (e.g., CT1 and CT2). AG1 and AG2 are 
the first and second most common responses (modes) for that scenario across all 
respondents from all states. State data headings indicate the first and second modes 
for that scenario within that state. Note that secondary state modes (e.g., CT2, ME2, 
MA2) are only shown when the number of respondents is greater than or equal to 2. 
Below the aggregate and state headings, below each scenario, are the abbreviations as 
shown in Table 5 that denote the LRE modal responses. These are shown both in 
aggregate and by state. The number of state respondents that selected each modal 
response is in the n row. Below the n is the ratio of state respondents selecting the 
modal response to the total number of useable respondent answers from that state for 
that scenario. In other words, the number of state respondents that chose this 
classification over the number of total state respondents for this scenario. The ratio can 
be seen as a measure of agreement within a state. (Note that some respondents 
categorized some scenarios as “don’t now” or provided no answer and are excluded 
from that scenario’s ratio.) Pink cells denote disagreement between the state score and 
aggregate score. Finally, when a state had the same number of respondents for both 
state scores 1 and 2 (bimodal) the number or responses are shown in bold italicized red 
text in the n row. 



Table G-1. Aggregate and State level coding consensus for LRE school age 
scenarios 


1) Steve’s parents removed Steve, a 7th grader, from public school and placed him in a private 
prep school in their district. Later Steve is found eligible for special education. Steve now 
receives 2 hours a day of pull-out services in the resource room at the private school. 


AG1 


AG2 




CT 1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


PrS 


21-60 


PrS 


<21 


PrS 




PrS 




PrS 


21- 

60 


21- 

60 


PrS 


21- 

60 




n 




4 


4 


3 




4 




5 


2 


2 


2 


7 




Ratio 


L 


4:9 


4:9 


3:4 




4:8 




5:8 


2:8 


2:6 


2:6 


7:8 




2) Jamie goes to the local high school in the morning for 15 hours a week. While there Jamie 
receives 5 hours a week of special education in a co-taught program. (The co-taught classroom 
has one special education teacher and one general education teacher. The general education 
teacher is the teacher of record for this classroom.) In the afternoon Jamie goes to a work study 
program at the local McDonald's for a total of 15 hours a week. 


AG1 


AG2 




CT1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


<21 


21-60 


< 21 




<21 




<21 




21- 

60 


<21 


< 21 




<21 




n 


7 




2 




6 




3 


2 


5 




8 




Ratio 


7:9 




2:4 




6:8 




3:6 


2:6 


5:6 




8:8 




3) A district places Holly at a residential school for students with sensory impairments. Holly lives 
close to the school, so she takes the bus to and from school as a day student. Assume this is a 
public residential school. 


AG1 


AG2 




CT 1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


PuS 


PuR 


PuS 


PuR 


PuS 




PuS 




PuS 


PuR 


PuS 


PuR 


PuS 




n 


7 


2 


3 




6 




5 


3 


4 


2 


4 
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Ratio 




7:9 


2:9 


3:4 




6:7 




5:8 


3:8 


4:6 


2:6 


7:8 




4) What is Holly's placement if the school is a private residential school? 


AG1 


AG2 




CT1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


PrS 


PrR 


PrS 


PrR 


PrS 




PrS 


PuR 


PrS 


PrR 


PrS 


PuR 


PrS 




n 


4 


2 


3 




5 


3 


4 


4 


4 


2 


8 




Ratio 


4:9 


2:9 


3:4 




5:8 


3:8 


4:8 


4:8 


4:6 


2:6 


8:8 






5) The district placed Pat at a private separate school for students on lEPs. Though still attending 
there part of the day, he is now transitioning back to his local high school. Currently his daily 
routine is to attend the private school from 9-12 then he is bussed to the public high school. In 
the afternoon he is in two general education classes with an instructional assistant who is 
responsible for Pat and another student with disabilities. He also attends one special education 
life skills class at the public high school. Each high school class is an hour long 


AG1 


AG2 




CT1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


> 60 


PrS 


> 60 


21- 

60 


> 60 




PrS 


> 60 


> 60 




>60 




>60 




n 


6 


2 


3 




4 


2 


5 




5 




5 




Ratio 


6:8 


2:8 


3:4 




4:8 


2:8 


5:7 




5:6 




5:7 




6) Jane is a parentally placed private school student with a Service Plan. During the first part of 
the year she received 1 hour of speech/week on a “pull-out” basis. The district decided to put her 
on an IEP in November but service levels and placement remain the same. 


AG1 


AG2 




CT1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


<21 


PrS 


< 21 


PrS 


<21 




<21 


PrS 


PrS 




na 


Na 


<21 




n 


5 


3 


3 




4 


3 


7 




na 


Na 


8 




Ratio 




5:8 


3:8 


3:4 




4:7 


3:7 


7:8 




Skipped 


8:8 




7) Matt has two periods in community job experience, two general education content courses with 
special education support as needed, one class in a resource room with a special education 
teacher on study skills, and one general education elective class (without special ed support). He 
has no other classes and the frequency and duration of all classes is the same. 


AG1 


AG2 




CT 1 


CT2 


ME1 


ME2 


MAI 


MA2 


NH1 


NH2 


NJ1 


NJ2 


NY1 


NY2 


<21 


21-60 


< 21 


21- 

60 


21- 

60 




21- 

60 




<21 


21- 

60 


<21 




<21 


21- 

60 


n 


3 


3 


3 




5 




4 


4 


5 




5 


3 


Ratio 




3:7 


3:7 


3:4 




5:7 




4:8 


4:8 


5:6 




5:8 


3:8 
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